Papers by Gerard De Melo
AutoMedEval: Harnessing Language Models for Automatic Medical Capability Evaluation (2025.acl-long)
Copied to clipboard
Xiechi Zhang, Zetian Ouyang, Linlin Wang, Gerard De Melo, Zhu Cao, Xiaoling Wang, Ya Zhang, Yanfeng Wang, Liang He
| Challenge: | Existing evaluation methods based on large language models (LLMs) are expensive and lack expertise due to limitations in human expertise. |
| Approach: | They propose an open-source automatic evaluation model with 13B parameters specifically engineered to measure the question-answering proficiency of medical LLMs. |
| Outcome: | The proposed model surpasses baselines in terms of correlation with human judgments. |
LLMs Cannot (Yet) Match the Specificity and Simplicity of Online Communities in Long Form Question Answering (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Recent years have positioned Large Language Models (LLMs) as powerful question answering (QA) tools, shifting users away from interacting in communities towards discourse with AI-driven conversational interfaces. |
| Approach: | They propose to use a QA preference dataset to fine-tune and align Large Language Models (LLMs) from more than 7.4 million submissions and 82 million comments from 2008 to 2022 in Reddit’s 15 largest finance communities. |
| Outcome: | The proposed framework improves on the social quality of the data, and the proposed framework is more accurate and more specific. |
Multi-Modal Bias: Introducing a Framework for Stereotypical Bias Assessment beyond Gender and Race in Vision–Language Models (2023.eacl-main)
Copied to clipboard
| Challenge: | Recent advances in self-supervised training have led to a new class of pretrained vision–language models. |
| Approach: | They propose a visual and textual bias benchmark to assess bias in self-supervised multimodal models using 3,800 images and phrases from 14 population subgroups. |
| Outcome: | The proposed model shows that it favors certain groups while maintaining the accuracy of the model. |
NextLevelBERT: Masked Language Modeling with Higher-Level Representations for Long Documents (2024.acl-long)
Copied to clipboard
| Challenge: | (large) language models struggle to process long sequences due to the quadratic scaling of the underlying attention mechanism. |
| Approach: | They propose a Masked Language Model operating on higher-level semantic representations in the form of text embeddings to solve this problem. |
| Outcome: | The proposed model outperforms larger embedding models on three types of tasks. |
GraphLSS: Integrating Lexical, Structural, and Semantic Features for Long Document Extractive Summarization (2025.naacl-short)
Copied to clipboard
| Challenge: | Graph-based methods for extracting documents have been popular, but they often require external tools or additional machine learning models to define graph components. |
| Approach: | They propose a heterogeneous graph construction for extractive summarization that defines two levels of information and four types of edges without any need for auxiliary learning models. |
| Outcome: | The proposed graph construction outperforms previous graph-based models on two datasets and is available on GitHub. |
Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Models (2025.findings-acl)
Copied to clipboard
| Challenge: | Recent work has shown that pruning can reduce model performance, but it can also lead to degradation in safety performance. |
| Approach: | They propose a hierarchical safety realignment approach to prune large vision-Language Models . they quantify contribution of each attention head to safety and restore neurons . |
| Outcome: | The proposed approach achieves significant safety improvements in LVLMs pruned post pruning. |
CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios (2024.emnlp-main)
Copied to clipboard
| Challenge: | Chinese medical large language models (LLMs) are underperforming on this benchmark, especially where medical reasoning and factual consistency are vital. |
| Approach: | They propose a benchmark with 14 expert-guided clinical scenarios to assess the medical ability of large language models across 7 pivot dimensions. |
| Outcome: | The proposed benchmark has been validated in several ways. |